Title: Erratic levelling fix
Author: Assassin
Version: Beta 0.20
Applies to: Secret of Evermore -- US version, German, French, Spanish, and English
            European versions
Tested on: US version, and to a much lesser degree, on the German, French, Spanish
           and English European versions

Contents:

        level-u.ips = the SoE (US) patch
        level-e.ips = the SoE (Europe - English) patch
        level-s.ips = the SoE (Europe - Spanish) patch
        level-f.ips = the SoE (Europe - French) patch
        level-g.ips = the SoE (Europe - German) patch
  
        Nlevel-u.ips = the SoE (US) anti-patch
        Nlevel-e.ips = the SoE (Europe - English) anti-patch
        Nlevel-s.ips = the SoE (Europe - Spanish) anti-patch
        Nlevel-f.ips = the SoE (Europe - French) anti-patch
        Nlevel-g.ips = the SoE (Europe - German) anti-patch

        The "Anti-patches" effectively remove this patch.

        readme.txt = this file

        soe-exp-per-level.txt = the experience the boy and the dog need to reach
                                each level.  also indicates all the levels the
                                dog might attain prematurely.  looks eerily like
                                a chart in Iron Knuckle's guide, but you're too
                                mesmerized to notice....

        level-up-orig.txt  = commented original code (US)
        level-up-fixes.txt = commented fixed code (US)

        level-up-orig-euro-english.txt  = commented original code (Europe - English)
        level-up-fixes-euro-english.txt = commented fixed code (Europe - English)

        euro-spanish-diffs.txt = Details everywhere that the European Spanish patch
                                 values differ from the European English version's.

        euro-french-diffs.txt = Details everywhere that the European French patch
                                values differ from the European English version's.

        euro-german-diffs.txt = Details everywhere that the European German patch
                                values differ from the European English version's.

        (The reason I have full code for only two versions is because in the
         patch-affected regions, the various European releases all differ from the
         USA one in roughly 30 bytes, but they only differ from each other in
         roughly 10.  Besides, the dog's level-up function is shifted back 16 bytes
         from USA to Europe, but none of the functions I included are shifted
         between European versions.)


ROM Addresses: All - F8480 thru F84FB, F8516 thru F8586
               SoE US - F86FB thru F876B
               SoE Europe - F86EB thru F875B

Functions added (existing code areas): All - 8F/8342 (23 bytes), 8F/8359 (21 bytes),
                                             8F/836E (24 bytes)
                                       SoE US - 8F/8527 (23 bytes), 8F/853E (23 bytes),
                                                8F/8555 (12 bytes)
                                       SoE Europe - 8F/8517 (23 bytes), 8F/852E (23 bytes),
                                                    8F/8545 (12 bytes)
Functions added (free space): None
Additional Square functions used: None

Urgency: Medium.  The fact that there are two bugs addressed certainly gives the patch some
         urgency.  The first bug can occur quite a bit (at 34 of the dog's 99 levels).  The
         second bug probably won't occur much -- aside from the Carltron battle -- unless
         you play through the game at lower levels.  Both bugs' effects are naturally
         remedied fairly quickly, and neither sway difficulty much one way or another.
         Besides, it's not like premature levelling is a physiological flaw, anyway.  It
         better ensures the survival of the species!

         If anything, the obvious nature of the bugs is what makes them noteworthy.  When
         players are levelling up the dog, they're bound to exclaim, "But didn't he just
         gain a level a minute or two ago?!"  When it happens again, they'll get spooked.
         Soon, they'll fear a clandestine government organization is compressing time in
         their household, and will slowly start to unravel.  Players are even more likely
         to raise an eyebrow if they see a character gain a level on two or more successive
         enemy defeats, especially when the later enemies were wimps.  Should they pop into
         the STAT menu between these level gains, an experience needed value of 4 billion
         will prompt a doubletake.

============================================================================================

The latest versions of all my patches can be found at either of these sites:

http://www14.brinkster.com/assassin17/
http://assassin17.home.comcast.net/

============================================================================================

TABLE OF CONTENTS

0. Description
1. Cause - First Bug
2. Cause - Second Bug
3. What my Patch Does
4. For Testers
5. FAQ
6. Revision History
7. Credits
Unnumbered: Copyright notice

____________________________________________________________________________________________

0. DESCRIPTION
____________________________________________________________________________________________

There are two distinct bugs here, so I'll explain them separately.

BUG #1:

Each character needs a certain amount of total experience to reach each level.
Unfortunately, a wrong instruction was used in comparing the dog's current experience to
the amount needed to attain its next level.  As a result, when the dog is level 39 or
higher, he'll sometimes gain a level sooner than he should (e.g. after having gained one
very recently).  This can happen on 34 of the dog's 99 levels.  However, things will
usually even out pretty quickly: he'll have to wait that much longer to level-up again, as
the bug doesn't alter his current experience.  The bug doesn't occur in reaching level 99,
so the total amount of experience needed to "max out" the dog is unchanged.

On very rare occasions, this bug can delay the dog's levelling as opposed to hastening it.
See the FAQ section for more details.

--------------

BUG #2:

If a defeated enemy gives a character (the boy or the dog) enough experience to make him
gain multiple levels, he'll still just gain one.  Then the character'll gain a level for
each subsequent enemy killed (even if it's a paltry mosquito or flower; it doesn't matter),
until he's at the proper level for his experience.  In the meanwhile, the (Experience)
"Needed" on the STAT screen will show up as some huge number (because the game thinks it's
negative, but whatever displays the number isn't programmed to handle negative values).
This bug results from the fact that the "Gain experience and level up if necessary" routine
is only called once for each enemy you defeat, and the level can only be boosted once per
call.

--------------

This patch fixes the first bug by making the game compare current and needed experience
for the dog in the same way as it does for the boy, who experiences no such bug.  It fixes
the second bug by allowing a single enemy defeat to result in as many level gains as
necessary.

____________________________________________________________________________________________

1. CAUSE - FIRST BUG
____________________________________________________________________________________________

In a nutshell, the problem is that when comparing the dog's current experience to the total
experience it needs to reach the next level, the game assumes that if the result of the
comparison (i.e. a subtraction involving two variables) is positive, the current experience
must have met or exceeded the value needed to level up.  Such an assumption is quite true
mathematically, but it's not always true in the game, due to how numbers are stored and
dealt with in computers.

Specifically, the game treats the subtraction as signed when it should be treated as
unsigned.

Read on for further explanation if you can stand horrible tedium.


To determine when they gain levels, the boy and the dog each have two variables:
their current experience, and the total amount of experience needed to reach the next level.
(I'll refer to those respectively as Current_Experience and Experience_Needed below.)

When the character's first variable meets or exceeds the second, they level up.  Simple.

However, each of these variables is 24-bit, and the SNES can't access more than 16 bits of
data in a single instruction.  Thus, two comparisons are needed to see whether the character
has met the experience for the next level.  And because the SoE programmers apparently
didn't want to switch back and forth between 8 and 16-bit mode, these 24-bit variables take
up 32 bits (though to my knowledge, the top byte is always zero).

The basic steps are (starting after we add the experience yielded by the enemy to our
current experience):

---------------------------------------

First for the boy:

1) Subtract the top two bytes of Experience_Needed from the top two bytes of
   Current_Experience.
   If the result is negative, we won't be gaining a level, so exit.
   If the result is positive and nonzero, we'll be gaining a level, so jump to step 3.

   (In either case, we can ignore the two bottom bytes -- Why?  Think of comparing two
    4-digit numbers.  If one is 04xx and one is 03yy, we already know the first number is
    greater than the second, so we don't *care* what xx or yy are.)

   Otherwise, the result is zero (i.e. the top two bytes match), so continue.

2) Subtract the bottom two bytes of Experience_Needed from Current_Experience.

   If the result cleared the Carry flag (i.e. a borrow was needed for the subtraction), we
   won't be gaining a level (as the complete Current_Experience variable is less than the
   complete Experience_Needed), so exit.  Otherwise, continue to Step 3, knowing that
   Current_Experience is greater than or equal to Experience_Needed.

3) Boost the character's level.

4) Set the character's Experience_Needed to the number necessary for the next level.


Repeat the process for the dog, except Step 2 looks like:

2) Subtract the bottom two bytes of Experience_Needed from Current_Experience.

   If the result is negative, we won't be gaining a level (as the complete
   Current_Experience variable is [supposedly] less than the complete Experience_Needed),
   so exit.  Otherwise, continue to Step 3, [supposedly] knowing that Current_Experience is
   greater than or equal to Experience_Needed.

---------------------------------------

The problem lies in the dog's Step 2.  See, because computers work with a finite space for
storing numbers, getting a positive result from A - B DOESN'T NECESSARILY MEAN that A >= B
(likewise, a negative result doesn't always mean A < B).

Let's say the dog is Level 42, and needs 558690d (08 86 62h) experience to reach level 43.
An enemy is killed, bringing the dog to 525382d (08 04 46h) current experience.  08 == 08,
so Step 1 continues to Step 2.  You and I know that 0446h - 8662h equals -7DE4h (or -33308
decimal), but the computer recognizes the result as a positive 7DE4h.  (A signed 16-bit
variable is only capable of representing the positive numbers 0 through 32767, and the
negative numbers -1 through -32768.  It can't "fit" -33308.  Read up on "Twos-complement"
numbers or arithmetic if this doesn't make sense to you.)

In other words, looking at the Sign flag means we're treating the subtraction that just
occurred as involving signed numbers, or as 0446h - -799Eh = 7DE4h (in decimal,
1094 - -31134 = 32228).

Since the difference of the bottom bytes is positive, the game assumes that
Current_Experience >= Experience_Needed, and boosts the dog's level.

The boy doesn't have this problem because the programmers correctly tested the Carry Flag
rather than the Sign flag.  When calculating unsigned A - B (which is what we want to do,
because there's no such thing as negative experience), Carry will always be cleared if
A < B, so we can rely on the flag regardless of the numeric result of the subtraction.

What about Step 1, which is performed for both the boy and dog?  It uses the same
problematic branch instruction as the dog's Step 2, but it shouldn't cause any trouble.
As I mentioned earlier, the topmost byte of Experience_Needed and Current_Experience is
always 0, meaning subtraction of the top two bytes will always look like: 00WX - 00YZ.

No matter what W, X, Y and Z are, we'll never be able to create a result that is "too large"
of a negative number to fit in a 16-bit variable.  Thus, Step 1 ought to be just fine.
(Though I go on the record as shunning it :P )

The cause of the bug really has to be oversight.  Square knew what they were doing with
the boy, but forgot to follow suit for the dog.

____________________________________________________________________________________________

2. CAUSE - SECOND BUG
____________________________________________________________________________________________

To quote Section 0:

"This bug results from the fact that the 'Gain experience and level up if necessary' routine
can only be called once for each enemy you defeat."

Either I'm exhausted from writing Section 1 or this bug is just that much simpler; I can't
find much reason to elaborate. :)

What the game currently does when you defeat an enemy is:

1) If the boy isn't at level 99 (or above):

   a. Add that enemy's experience to the boy's experience, provided the boy's in the party.
   b. If the boy has enough experience to reach the next level, increase his level by one
      (and update the experience needed variable, boost any stats as needed, and queue
      a level-up message to be displayed).

2) If the dog isn't at level 99 (or above):
  
   a. Add that enemy's experience to the dog's experience, provided the dog's in the party
      and has nonzero HP.
   b. If the dog has enough experience to reach the next level, increase his level by one
      (and update the experience needed variable, boost any stats as needed, and queue
      a level-up message to be displayed).


The bug likely resulted from an oversight; Square probably just forgot that there can be
cases where you should gain multiple levels from defeating a single enemy.  They are
limited, after all.

An aside: There's nothing wrong with the code that displays (Experience) "Needed" on the
STAT menu; it's just a victim of this bug.  Since I mentioned the menu only briefly in
Section 0, I'll go into more detail as to what's probably causing the screwy display.  Let's
say your character is level 30.  They kill an enemy with enough experience to bring them to
level 32, but they only reach level 31 due to the bug.  To get the experience needed value
shown on the menu, the game subtracts your current experience from the total experience
required to reach the next level -- i.e. level 32.  Because you already have more than
enough experience for level 32, that value is negative.  And because Square rightly never
anticipated a negative value here, it just shows up as a realllly big positive value rather
than a small negative one.  (My guess is a 32-bit integer was used for the "Experienced
needed" calculation and display.  If you treat such an integer as unsigned, it can have a
value from 0 to +4294967295.  If you treat it as signed, it can range from -2147483648 to
+2147483647.  The game does the former for the display, so when it calculates -5000
experience needed, positive "4,294,962,296" is what shows up.)

____________________________________________________________________________________________

3. WHAT MY PATCH DOES
____________________________________________________________________________________________

FOR BUG #1:

Tests the Carry flag rather than the Sign flag in the dog's Step 2 (see Section 1).  Now the
dog will level-up correctly, as the boy already does, and no longer gain levels prematurely.

Also, optimizations conveniently changed Step 1's branch instruction for both characters. :P

----------------

FOR BUG #2:

Allows a character to level-up multiple times for each enemy killed.  Now the steps shown
in Section 2 become:

1) If the boy isn't at level 99 (or above):

   a. Add that enemy's experience to the boy's experience, provided the boy's in the party.
   b. If the boy has enough experience to reach the next level, increase his level by one
      (and update the experience needed variable, and boost any stats as needed). 
      Otherwise, skip to Step 1d.
   c. If the boy isn't already at (or above) level 99, go to Step 1b again.
   d. If the boy gained at least one level from this enemy, queue a level-up message to be
      displayed, showing the boy's new level.  Otherwise, just move onto Step 2.

2) If the dog isn't at level 99 (or above):
  
   a. Add that enemy's experience to the dog's experience, provided the dog's in the party
      and has nonzero HP.
   b. If the dog has enough experience to reach the next level, increase his level by one
      (and update the experience needed variable, and boost any stats as needed).
      Otherwise, skip to Step 2d.
   c. If the dog isn't already at (or above) level 99, go to Step 2b again.
   d. If the dog gained at least one level from this enemy, queue a level-up message to be
      displayed, showing the dog's new level.  Otherwise, exit.

____________________________________________________________________________________________

4. FOR TESTERS
____________________________________________________________________________________________

Any testing is appreciated, as this is only my second patch for SoE.  Also, I tore up three
different functions, so it's possible I flubbed something.  While I tested the USA version
pretty well, I don't have any saves for the European versions, so I wasn't able to do much
other than verify that the patches don't interfere with reaching level 4 against jungle
flowers and mosquitoes. :/  In other words, I haven't tested whether those patches solve the
existing bugs, but I did make sure they don't introduce any new ones.  More scrutiny on the
European patches would be *great*.

Please provide feedback on the GameFAQs Secret of Evermore message board:  
http://www.gamefaqs.com/console/snes/data/588645.html

or Mnrogar's Den:
http://www.mnrogar.com/

Thanks!

____________________________________________________________________________________________

5. FAQ
____________________________________________________________________________________________

Q: At what levels does the first bug occur anyway?

A: Plenty.  See soe-exp-per-level.txt for which ones.  The problem doesn't happen at lower
   levels because the experience needed to reach one level is closer to that needed for
   the next.


Q: You talk of the first bug making the dog level up too fast.  What about the reverse
   situation: the dog thinks it doesn't have enough experience to reach the next level, even
   when it does?

A: That's a theoretical possibility.  To see it, you'd need a really large value in the
   bottom two bytes of Current_Experience, and a really small one in the bottom two bytes of
   Experience_Needed.  For example, a Current_Experience of 0C8B00h and an Experience_Needed
   of 0C089Ch could do it.  However, a scenario like this is very unlikely to unfold,
   because you _gradually_ gain experience.  It's quite probable Current_Experience reached
   a smaller value like 0C0D00h a long time before it got to 0C8B04h, thus triggering a
   level-up, and avoiding the bug.

   To see the bug, you'd need an enemy giving a ton of experience: in the neighborhood of
   32768 points.  Most enemies don't come close to that (one gives 10000, and another gives
   8000).  Carltron gives 100000, so some ways to see the bug with him are: to be at
   level 47 (and have between 738308 and 751967 experience) when the robot appears, be at
   level 62 (between 1779009 and 1800543 experience), be at level 64 (between 1968355 and
   1997951 exp), etc.

   Like the first manifestation of the bug, this one's effects will be naturally remedied
   fairly quickly.  Once the dog in the example two paragraphs ago hits 0Dxxxxh experience,
   it'll gain that belated level.


Q: Now that the second bug is fixed, what will happen when a single enemy gives me enough
   experience to gain multiple levels?

A: You will gain all those levels.  However, there will still just be one level-up message
   per character, which looks like: "[Character name] reaches level [highest level]".

   Initially, I had tried to allow multiple messages, as that would've been more
   informative, and 2 out of 3 people polled on the GameFAQs SoE board favored it.  Also,
   it seemingly required the least code changes; because the code to handle level-up
   messages is part of the level-up function, there'd naturally be one message queued per
   level gained.  However, a couple problems arose:

   1) If you get 4 level-up messages with the boy and 4 with the dog against Carltron's
      Robot (which happens with the Level 36/35 save I used), the dog's last message can
      actually occur *after* the angry butler starts his post-battle conversation.  It's
      out of place, but it doesn't make the convo unreadable in any way.  This didn't
      bother me too much.

   2) Now the big one: if you get more than 8 total level-up messages -- possible with
      Carltron at levels in the low 30s or beneath -- only some of them will be displayed.
      Often none.  It's due to a limitation in the size of a pointer buffer the game uses.

      I don't know nearly enough about the game to enlarge the buffer or to work around this
      issue, so this problem's quite the stickler.


   If it seems like I'm putting too much emphasis on one battle, well, Carltron is far and
   away the most likely enemy to make you gain multiple levels in the first place. :)

   Hence the way the patch does it now.  I'm perfectly comfortable with it, as it conveys
   the character's new level after defeating the enemy, which is the most important piece
   of information.


Q: How much experience do the various enemies give, anyway?  I'd like to know who can make
   me gain more than one level at a time.

A: See my Monster Statistics Guide -- available on my websites and GameFAQs -- for the
   experience yielded, and soe-exp-per-level.txt -- included in this archive -- for the
   experience needed.  As far as I can tell, only a handful of monsters can induce multiple
   level-ups.  Obviously, these are much more likely to occur if you're at lower levels, so
   consult the Low Level Walkthrough by josher1212 (once again, on GameFAQs) to gauge the
   lowest level you can be at when fighting a given foe.

   Carltron's Robot gives a hefty 100000 experience, which means he can cause multiple gains
   at levels as high as 65 (though they're rather inconsequential).  Coleoptera, with 10000
   experience, can do so up to level 21.  I'll let you look for other candidates.

____________________________________________________________________________________________

6. REVISION HISTORY
____________________________________________________________________________________________

Version 0.20 : January through May 2005

  - January: Figured out the first bug's cause sometime around then.  Dove into the Readme,
    completing most of Section 1 and part of Section 2 by mid January.
  - mid March: worked more on Readme.  Made the experience per level list.  Happily geared
    up for a 1-byte patch that was identical on all versions of the game.  What a fool I
    was.
  - late March through mid April: Studying the code led me to discover the second bug, if
    only on paper.  Fixing it required me to add to function 8F/827A, so I set about making
    the revised function fit in the original space, and got pretty close thanks to
    Imzogelmo.  Virus973 and I both confirmed the bug indeed shows up in gameplay.
    Resorted to invading the boy's and dog's level-up functions to accomodate my added
    code.  Unlike 8F/827A, which is nigh identical across game versions, these functions
    have all sorts of differences in the addresses used; no two nationalities of the ROM are
    the same.  Such a far cry from earlier this month. :'(  Optimized these functions for
    space, with Imzogelmo once again lending a hand.
    Wrote the USA commented assembly files and made the USA patch.  Tested that patch.
    Decided multiple level-up messages per character per monster wasn't going to work out,
    so I started retooling the code to only display the highest level reached.
  - mid to late April: The patch was becoming a bit of a pain in the arse, so I took a break
    from it to write the SoE Monster Statistics Guide. :)
  - late April to early May: Finished writing new code, which would just display,
    "[Character name] reaches level [highest level]" after a character gains multiple levels
    from defeating a single enemy.  Made the USA patch.  Tested that.  Finished the
    commented assembly files for the USA version.  Wrote the lists of differences between
    European versions.  Made the commented assembly files for the European English version.
    Made the various European patches.  Tested those, albeit incompletely.  Put finishing
    touches on Readme.
  - May 9: Released patch.

____________________________________________________________________________________________

7. CREDITS
____________________________________________________________________________________________

  - Iron Knuckle for his/her excellent FAQ on the game, which reminded me of the first bug's
    existence.  I stole the format for the experience per level list from this (though mine
    was generated by a program that read the ROM), and the guide's descriptions of the first
    bug were of aid in writing mine.

  - Imzogelmo for a few fine optimizations.  That's right, kids, he can school me at coding
    for a game he hasn't even PLAYED!  Give the NEPROMR a look at
    http://www.angelfire.com/al2/imzogelmo/patches.html

  - Virus973 of the GameFAQs _Secret_of_Evermore_ board for confirming seeing the second bug
    in the actual game.  Also for responding to my poll on how the level-up messages should
    be displayed.

  - Mnrogar for responding to my poll on how the level-up messages should be displayed.  Also,
    he runs a fine little website:
   
    http://www.mnrogar.com

    where he's even kind enough to acknowledge SoE project releases as newsworthy. ;P

  - Minako Aino for responding to my poll on how the level-up messages should be displayed.

______________________________________________________________________________________________

Secret of Evermore copyright 1995 Squaresoft.
This readme and all other files in the archive (as listed above)
  copyright 2005 Assassin.
All rights reserved.

